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Abstract 


The aim of this research is produce a set of PISA-like mathematics task with Indonesia 
natural and cultural heritage as context which are valid, practical, to assess students’ 
mathematics literacy. This is design research using type of development research with 
formative evaluation. A total of 20 students of SMP Negeri 1 Palembang. Beside, 10 experts 
were involved in this research to assess the feasibility of prototyping in terms of content, 
context and language. Walk through, documentation, questionnaire, test result, and interviews 
are way to collect the data. This research produced a PISA-like math task is as many 12 
category of content, context, and process valid, practical and has potential effect. The validity 
came empirical evaluation of validation and reliability testing during small group. From the 
field test, we conclude that the tasks also potentially effect to the students’ mathematical 
literacy in activating the indicators of each Fundamental Mathematical Capabilities. 

Keywords: development research, PISA task, mathematics literacy, fundamental 
mathematical capabilities 


Abstrak 


Tujuan dari penelitian ini adalah menghasilkan soal PISA dengan konteks warisan alam dan 
budaya Indonesia yang valid dan praktis untuk menilai kemampuan literasi siswa. Penelitian 
ini adalah jenis penelitian design research dengan tipe development research. Penelitian ini 
menggunakan alur formative evaluation. Subjek dalam penelitian ini adalah 20 siswa kelas IX 
SMP Negeri 1 Palembang. Disamping itu, terdapat 10 dosen ahli yang terlibat dalam 
penelitian ini, baik dari dalam maupun luar negeri. Para ahli tersebut menilai kelayakan soal 
dari segi konten, konteks, dan bahasa. Proses pengumpulan data diperoleh dengan cara walk 
through, documentation, questionnaire, test result, dan interviews. Uji validitas dan 
reliabilitas soal dilakukan pada tahap small group. Pada tahap field test, diperoleh bahwa soal 
PISA dengan konteks warisan alam dan budaya Indonesia mampu memunculkan kemampuan 
literasi siswa berupa kemampuan dasar matematika. 

Kata kunci: development research, soal PISA, literasi matematika, kemampuan dasar 
matematika 

Badan Standart Nasional Pendidikan (BNSP, 2006) written that mathematics subject in the junior high 
school has several purposes. Students can understand the concept of mathematics, this means that they 
can be explain the relationship between concept and apply the concept or algorithm appropriately. 
Second, students can use the pattern and characteristics of the mathematics reasoning, making 
generalization in mathematical manipulation, compile evidence, and explain ideas and mathematical 


statement. 


Third, solving the mathematical problems which include the ability to understand the problems, 
designing mathematical models, and completing the model and interpret the solution. Students should 
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communicate the ideas with symbol, tables, diagrams, or other media to clarify the issue. The last, the 
purpose of mathematics subject in the junior high school is to make student have respect for the 
usefulness of mathematics in the life. Base on the mathematics purpose, the can be seen that the 
curriculum has noticed the aspect of mathematical literacy. 

PISA (Programme for International Student Assessment) is an international organization that 
evaluation the mathematical literacy skills of students in the world. The Organization for Economic 
Co-operation and Development (OECD, 2013) explain that the focus of PISA is the emphasis on 
students’ skills and competencies obtained from school and can be used in everyday life and in 
various situations. PISA-like mathematics task requires students to have higher thinking skills, by 
combining their knowledge in solving mathematical problems (Wardhani, 2004). 

But, in the reality the mathematical literacy of Indonesia students is lower than the other 
country. It’s because they are not accustomed to work on the mathematical problems with high- 
thinking skills (Shadiq, 2007). In the other hand, lulaiha (2001) explained that the most of Indonesia 
students are not familiar with the PISA problems because the PISA problems using the realistic 
context. Jupri, et al (2013) adds that there are several things that make the students fail in PISA survey 
which is when the students formulate mathematical problems into formal form and when they have 
found the results, they don’t proceed to the next step. Therefore, this research aims to produce a set of 
PISA-like mathematics task which are valid, practical, and has potential effect to the development of 
student mathematical literacy. 

The definition of mathematical literacy in PISA 2012 is the capacity of students to formulate, 
employ, and interpret mathematics in a variety of context. Kusumah (in Aini, 2013) explain that 
mathematical literacy contains the ability to construct a set of questions, formulate, solve, and 
interpret the problems based on the context. Niss ( in Aini, 2013) also express that mathematical 
literacy have several indicators, those are : (1) reasoning and mathematical thinking; (2) mathematical 
argumentation; (3) mathematical communication; (4) modeling; (5) submission of problems solving; 
(6) representation; (7) symbol; and (8) media and technology. 

Mathematical literacy starts from realistic problems, which are categorized into category of 
context and content. The mathematical literacy process start from identifying the realistic problems 
and formulate the problem mathematically based on the concepts and relationship inherent in the 
problem. After getting an appropriate mathematical form of the problem, the next steps is to employ 
certain mathematical procedures to obtained mathematical results, which then interpret those back 
into the initial problems. 

METHOD 

This study is design research using type of development study. This study concern with 
iterative development using the formative evaluation in various consumer (Plomp & Nieveen, 2007). 
The formative evaluation contained in this research consisted of preliminary stage and prototyping 
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phase which includes self-evaluation, expert reviews and one-to-one, small group, and a field test 
(Zulkardi, 2002; Tessmer, 1993). 

The development process started form preliminary steps by grasping, the concept related to 
development of mathematical literacy task and used it to design an initial prototype. Afterwards, this 
prototype was then self-evaluated before going into the next steps. In expert review, ten expert were 
involved to validate the task in term of content, construct, and language. For one-to-one phase, 4 
students evaluated particularly on how they understand the information, for example, picture, or 
phrase, the task and not focus on how they answer the task. 

These result gave significant suggestion and revising the task so that those could be re¬ 
evaluated in small group. The small group phase involved 10 students with various academic abilities 
to solve the task in 75 minutes. Here, we firstly obtained the data of how students perform in solving 
the task because by analyzing the variety of students answer. We used this data to assess students real 
performance in field test. The field test was involving 20 students of grade IX form SMP Negeri 1 
Palembang. 

RESULT AND DISCUSSION 
Developing Tasks 

In the preliminary stage, we conducted several steps; (1) examined the literature on developing 
mathematical task, framework of the PISA 2015, the relationship between the current curriculum and 
the PISA survey, (2) designed an initial prototype comprising a set of PISA-like task and its scoring, 
(3) determined the validators, (4) determined the research subject. At the stage of self-evaluation, we 
examined the initial prototype resulting prototype 1. 

The prototype 1 then evaluated by validators or expert review and also through one-to-one 
phase that involving students. In the expert review phase, prototype 1 was assessed and evaluated by 
10 experts, namely Prof. Kaye Stacey and Dr. Ross Turner, the mathematics expert group of PISA, 
the PMRI lecturers, Prof. Dr. Ahmad Fauzan, Prof Dr. Ipung Yuwono and Dr. Yenita Roza, 
Mathematics teachers, Nurjannah, M.Pd and Nadya Husenti, M.Pd, and the last Dr. Ariyadi Wijaya, 
Kamaliyah, M.Pd, Ni Luh Sakinah M.Pd, Ahmad Wahidul Kohar, M.Pd as PISA researcher. These 
expert reviewed the prototype 1 in terms of the content, the construct and the language. Afterwards, 
the researcher conducted a test to 4 students SMP Negeri 1 Palembang grade IX individually (one-to- 
one). The following is one of tasks example on this situation. 



Oktiningrum, Zulkardi, & Hartono, Developing PISA-LIKE Mathematics Taks ... 


4 


Before Revision 


UNIT 3: BATIK 

Basic competence: solving problems by estimating the unknown quantity by using graphics, 
algebra, and aritmatics. 

7. Ana was ordered to buy Ram Batik in Pasar Klewer, a traditional market in Solo. After 
surveying the merchants in the market. Ana was interested to buy in the 2 shops having 
attractive offers. Toko Indrajaya and 
Toko Abadi. 

• Toko Indrajaya -> selling kain 
Batik in Kilo with the price Rp 
75000/Kg. For 5 Kg or more 
purchase, the price is Rp 70000/Kg. 

(Per-Kilo= the material is S-12 
meters in length, and 1,15 meters in 
width, based on the thickness and 
t hinne ss of the material). 

• Toko Surya Abadi -> selling kain batik per sheet with the price Rp 35000 - 45000 per 
sheet (depends on the the thickness and thinness of the material), the size is 115 x 200 
cm. 

Which shop that Ana should choose? Give your reason! 

8. If Ana is ordered to buy Kain Batik for her office uniform, consisting of 10 adult female 
and 15 adult male (note: all workers have a normal body size). Determine the minim um 
outgo to buy the uniform material based on the shop that Ana has chosen. 

Figure 1. Task before revision 



Table 1. Comments from Expert and Students on number 7 


Validation 

Comments/Responds 

Revision 


The question is not clear. 

• Changing the 


“This is not clear, or not sufficient 

question with the 


information. Does it mean that for 

realistic question 


the thickest material, 8 meters 

• Using different 


weighs one kilo, and for the 

context 

£ 

thinnest material, 12 meters 


<D 

i-i 

weights one kilo? If so, please state 


<D 

Ph 

X 

this more clearly, then I think the 


W 

problem does have a solution in its 

current form (ie, you can disregard 

most of my other comments) 

Change the question with “ Which 

store should be chosen Ana in 
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order to obtain a cheaper price?" 



I don’t know how to solve this 



problem, because for me is so 


Ch 

<D 

complicated problem 


CO 

This question need logical 

thinking, I can solve it by my self 


Table 2. Comments from Expert and Students on number 8 

Validation 

Comments/Responds 

Revision 


The question is not clear. 

• Changing the 


“I infer that you are mostly 

question with the 


interested in the reasoning 

realistic question 


followed by the student to make 

• Using different 


this decision. You should frame 

context 

£ 

your question to make it clear that 


<D 

’> 

CD 

this is your objective. For example, 


l-i 

Qh 

X 

W 

you could say “Give a detailed 

mathematical argument as to why 

Ana should choose Toko 

Indrajaya.” 



Problem impressed related to the 

previous problem 



The question no 7 and no 8 


e 

interrelated 





CO 

The answer of number 7 influence 



the answer of number 8 



After obtaining some suggestion from both experts and students, the prototype 1 was revised 
become prototype 2. The following is the revised task based on the Table 1 and Table 2. 
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After Revision 


Batik 

7. Look the motif Batik Cirebon beside! 

Determine the 100 th colour of the Batik... 

Baris 1<- 

8. SMP Negeri 1 Palembang, plans to make batik uniforms for students. They choose Batik 
Jogja, Motif Batik Songket and Batik Solo. Students were asked to choose the batik. From 

2 . . 5 

the calculations, - choose Batik Jogja, — choose Songket and Batik motif as much as 440 

students choose Batik Solo. Determine the total of all voice collected and batik which has the 
highest number of votes? 



Figure 2. Task 1 after revision 

Start from the result of expert review and one-to-one phase, the prototype 1 become prototype 
2. After that, prototype 2 tested to small group consisting 10 students from ninth grader. The results 
show that the task has coefficient of high reliability of 0,70 but some task were empirically invalid. 
Therefore, the researcher reviewed each items developed primarily on the invalid task for discarded, 
maintained with revisions, or retained without revision. This decision was based on the result of 
activities: (1) giving a questionnaire that ask students opinions regarding the task they have did; (2) 
examining the distribution of student’s answers; and (3) interviewing the students in the small group 
to investigate whether the student were not able to solve the task in the absence of the aid scheme or 
in the matter of readability issues. 

The result of this evaluation resulted in prototype 3 which was then used in a field trial test 
involving 20 students in order to know the potential effect to the task as well as to assess the 
mathematical literacy of students when they solve the mathematics PISA problems that use Indonesia 
natural and cultural heritage as context. 

Potential Effects of the Task 

After the students completed work on the task, the researcher gave questionnaires to all the 
students, and interviewed 4 of them to obtain the data about the potential effect. The student responses 
regarding the questionnaire is shown below. 



7 


Journal on Mathematics Education, Volume 7 , No. 1, January 2016, pp. 1-10 


Tabel 3. Students Response in Activating FMC 


No 

Activated Fundamental Mathematical Capabilities 

Response of 

students 

1 

Mathematizing 

18 % 

2 

Communication 

31% 

3 

representation 

50% 

4 

Devising problems solving strategies 

45% 

5 

Reasoning and argumentation 

91% 

6 

Use of formal / symbolic language 

45% 


Based on the Table 3, 6 of 7 Fundamental Mathematical Capabilities to be use of students in 

solving the prototype 3. From the table, it also 
can be seen that the reasoning and argumentation 
were recognized by most students in solving 
problems. And the mathematizing were least used 
by students. Additionally, the table below show 
the response of students after working the task. 


Table 4. Students Response on Prototype 3 


How interesting are the task? 

Response of students 

I am interested and serious in working all the tasks 

44% 

I am only interested and serious in working certain tasks 

55% 

I am not interested in the tasks at all 

1% 


<*> 


dtvpercecv' ^ mt> c 


Jiro 'V OS *9“ 

MW* otonun tw' 'eWn'uw. . 


As a proof how students activate their FMC into mathematical process: formulate, employ, 
interpret in solving the tasks, the following are examples of students’ work on task 10.The aim of the 
task is to know the students ability to estimate the area of field. (See Fig 3). 


KM 



(a) 


(b) 
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(c) (d) 

Figure 3. The students answer on task 10 


Form the figure 3 show that students have different ways to answers the task. Figure 3 (a) and 
(b) showed that students copying the picture in the same size and compare the picture. He found that 
the picture c have the large area than the other. Different way with (a) and (b), the figure 3 (c) showed 
that students compare the picture with small unit. And figure 3 (d) indicate that he use the some way 
with (a) and (b), but (c) copying the picture in the paper with the same size. After that, he compare the 
picture. All of the answers show that students using the various strategy to solve the problems. 

CONCLUSION 

This study produced set of PISA-like mathematics task with Indonesia natural and cultural 
heritage which valid and practical. Based on the result, prototype 3 has potential effect to assess the 
mathematical literacy of students, it shown by the student answers. Others indication of this effect are 
also seen from their seriousness and interesting when solving the task. Lastly, suggest teachers and 
other practitioners to use the task from study as tools in designing PISA problem based learning. 
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